Refactor GHAs #124

iowillhoit · 2024-08-01T19:08:38Z

Refactors a lot of our GHAs to pass inputs through the env.
A lot of other random cleanup. I have added a lot of comments on this PR to help with any confusion.

@W-16223935@

iowillhoit · 2024-08-01T21:00:19Z

.github/actions/ctcOpen/action.yml

@@ -59,5 +59,7 @@ runs:
        SF_CHANGE_CASE_TEMPLATE_ID: ${{ inputs.SF_CHANGE_CASE_TEMPLATE_ID}}
        SF_CHANGE_CASE_CONFIGURATION_ITEM: ${{ inputs.SF_CHANGE_CASE_CONFIGURATION_ITEM}}

-    - run: echo "case id is ${{ steps.ctc.outputs.ctcId }}"
+    - run: echo "[INFO] Change Case ID is:\ $STEPS_CTC_CTCID"


Note: You need to escape the space after a : if you want it to be valid yaml. Or put it on a new line...

- run: | echo "[INFO] Change Case ID is: $STEPS_CTC_CTCID"

iowillhoit · 2024-08-01T21:01:53Z

.github/actions/generateOclifReadme/action.yml

@@ -46,7 +46,9 @@ runs:
        yarn tsc
        yarn oclif readme \
          --no-aliases \
-          --version ${{ steps.next-version.outputs.tag }} \
+          --version "$STEPS_NEXT_VERSION_TAG" \
          ${{ inputs.multi == 'true' && '--multi' || '' }} \


Expressions that render values that don't include the input are fine.

iowillhoit · 2024-08-01T21:06:55Z

.github/actions/get-json-property/action.yml

+            }
+          } catch(err) {
+            core.setFailed(err)
+          }


Had issues reading nested values using jq and env vars. Rewrote this in javascript.

Passing:

package.json version (run)

package.json devDependencies.@salesforce/dev-scripts (run)

Failing:

package.json versionzzz (run)

package.jsonssss version (run)

iowillhoit · 2024-08-01T21:08:37Z

.github/actions/renameMacPkg/action.yml

@@ -1,22 +0,0 @@
-# This action is only needed as long as the developer site and other docs are linking to the old sfdx.pkg file and the mac signing job is only signing the old file as well.
-# It can be deleted once those are updated to use the new name, sfdx-x64.pkg.


This is no longer being used

iowillhoit · 2024-08-01T21:09:51Z

.github/workflows/automerge.yml

-        run: sf-release dependabot:automerge --merge-method ${{ inputs.mergeMethod}} --max-version-bump ${{ inputs.maxVersionBump}} --owner $GITHUB_REPOSITORY_OWNER --repo $(basename $GITHUB_REPOSITORY)
-      - name: run automerge command
-        if: ${{ inputs.skipCI }}
+        run: sf-release dependabot:automerge --merge-method "$INPUTS_MERGE_METHOD" --max-version-bump "$INPUTS_MAX_VERSION_BUMP" --owner $GITHUB_REPOSITORY_OWNER --repo $(basename $GITHUB_REPOSITORY) ${{ inputs.skipCI && '--skip-ci' || '' }}


This diff is confusing. I turned two steps into one using a "ternaryish"

iowillhoit · 2024-08-01T21:11:24Z

.github/workflows/ctcClose.yml

@@ -16,8 +16,6 @@ jobs:
    runs-on: static-ip-ubuntu-runners
    steps:
      - uses: actions/checkout@v4
-        with:
-          token: ${{ secrets.SVC_CLI_BOT_GITHUB_TOKEN }}


No idea why we would need our token for this workflow. The GH_TOKEN generated for the GHA run should work fine.

iowillhoit · 2024-08-01T21:12:18Z

.github/workflows/ctcOpen.yml

@@ -12,14 +12,14 @@ jobs:
    runs-on: static-ip-ubuntu-runners
    steps:
      - uses: actions/checkout@v4
-        with:
-          token: ${{ secrets.SVC_CLI_BOT_GITHUB_TOKEN }}


Same here, GH_TOKEN should work

iowillhoit · 2024-08-01T21:14:18Z

.github/workflows/devScriptsUpdate.yml

      - uses: salesforcecli/github-workflows/.github/actions/get-json-property@main
        id: packageVersion
        with:
          path: "package.json"
-          prop_path: 'devDependencies["@salesforce/dev-scripts"]'
+          prop_path: 'devDependencies.@salesforce/dev-scripts'


This changes is because I rewrote this action using javascript. I could not find any other uses of this action that used a nested value.

iowillhoit · 2024-08-01T21:14:52Z

.github/workflows/devScriptsUpdate.yml

+        env:
+          STEPS_PACKAGE_VERSION_PROP: ${{ steps.packageVersion.outputs.prop }}
+
+      - run: echo "[INFO] shouldUpdate value will be:\ ${{ !endsWith(steps.packageVersion.outputs.prop, steps.version-info.outputs.version) }}"


Expression will render to either true or false

iowillhoit · 2024-08-01T21:15:57Z

.github/workflows/devScriptsUpdate.yml

      - run: npm install -g yarn-deduplicate
      - run: yarn upgrade @salesforce/dev-scripts@latest
+      # TODO: What does this mean? Would yarnInstallWithRetries work here?
      # this may fail because that's how dev-scripts does things


Two open questions here (TODO), would appreciate some feedback.

TODO: Do we need this? Isn't that why we use npx?

Not sure why, yarn-deduplicate can be executed as a global binary. I can remove the call via npx.

TODO: What does this mean? Would yarnInstallWithRetries work here?

I'm pretty sure it's because of sf-install, when it updates dependencies it throws (exit code 1) and ask you to re-run yarn install, so the first yarn install continues on error to allow the second one (if the second one fails then the workflow exits):
https://github.com/forcedotcom/dev-scripts/blob/6bd069d98d14e8d1bfd629c720b8e5c87b8a45ed/bin/sf-install.js#L15

iowillhoit · 2024-08-01T21:17:48Z

.github/workflows/externalNut.yml

      - name: NUTs with ${{ inputs.attempts }} attempts
        uses: salesforcecli/github-workflows/.github/actions/retry@main
        with:
          max_attempts: ${{ inputs.attempts }}
          command: ${{ inputs.command }}
          retry_on: error
        env:
+          TESTKIT_EXECUTABLE_PATH: ${{ inputs.sfdxExecutablePath }}


I deleted the step above that sets this. I think setting it here on the env will work exactly the same.

the previous step was supposed to set the env var only if inputs.sfdxExecutablePath was set, I'm not sure if this is equivalent since the env var could be set to an empty string or undefined and make testkit search for an invalid CLI bin, I'll test on QA to confirm.

iowillhoit · 2024-08-01T21:18:30Z

.github/workflows/npmPublish.yml

      - name: Is published
        id: is-published
        run: |
-          RESPONSE=$(npm view .@${{ inputs.githubTag }} version --json --silent || echo "Not published")
+          RESPONSE=$(npm view .@$INPUTS_GITHUB_TAG version --json --silent || echo "Not published")


I tested this locally to make sure the env var is evaluated correctly 👍

iowillhoit · 2024-08-01T21:22:39Z

.github/workflows/nut.yml

      - name: NUTs with ${{ inputs.retries }} attempts
        uses: salesforcecli/github-workflows/.github/actions/retry@main
        with:
          max_attempts: ${{ inputs.retries }}
          command: ${{ inputs.command }}
          retry_on: error
        env:
+          TESTKIT_EXECUTABLE_PATH: ${{ inputs.sfdxExecutablePath }}


Same here (see comment above)

iowillhoit · 2024-08-01T21:23:28Z

.github/workflows/packUploadMac.yml

@@ -20,28 +16,39 @@ on:
 jobs:
  macos:
    env:
-      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
-      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}


I moved the AWS secrets to only the steps that need them. Otherwise they will be set on the env for the entire job.

iowillhoit · 2024-08-01T21:24:08Z

.github/workflows/packUploadMac.yml

@@ -20,28 +16,39 @@ on:
 jobs:
  macos:
    env:
-      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
-      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
-      SFDX_HIDE_RELEASE_NOTES: true


I moved this to an org level variable https://github.com/organizations/salesforcecli/settings/variables/actions

iowillhoit · 2024-08-01T21:24:48Z

.github/workflows/packUploadMac.yml

      - uses: salesforcecli/github-workflows/.github/actions/yarnInstallWithRetries@main
-      # todo: download the macos tarball and pass that to the oclif pack:macos command


We decided this did not provide any real benefit and already never'd a ticket

iowillhoit · 2024-08-01T21:26:33Z

.github/workflows/packUploadMac.yml

-      cli:
-        type: string
-        required: true
-        description: only needed if upload. sfdx or sf


No more sfdx 🎉 These inputs were removed here

iowillhoit · 2024-08-01T21:28:17Z

.github/workflows/packUploadWindows.yml

@@ -1,10 +1,6 @@
 on:


Changes in this file are the same as packUploadMac.yml. See those comments

iowillhoit · 2024-08-01T21:28:54Z

.github/workflows/publishTypedoc.yml

@@ -7,36 +7,41 @@ on:

 jobs:
  publish:
-    # todo: parameterize this as input if anyone ever needs ip-restricted runner


Not a priority

iowillhoit · 2024-08-01T21:30:04Z

.github/workflows/stampyUpload.yml

-      clipkg:
-        type: string
-        description: npm package name for cli
-        default: "@salesforce/cli"


No more sfdx 🎉 These inputs were removed here

iowillhoit · 2024-08-01T21:31:38Z

.github/workflows/tarballs.yml

      SF_DISABLE_TELEMETRY: true
    runs-on: ubuntu-20-8core
    steps:
      - uses: actions/checkout@v4
-        with:
-          token: ${{ secrets.SVC_CLI_BOT_GITHUB_TOKEN }}


Again, not sure why we would need our token here

iowillhoit · 2024-08-01T21:33:08Z

.github/workflows/stampyUpload.yml

+          STAMPY_UNSIGNED_BUCKET: ${{ secrets.STAMPY_UNSIGNED_BUCKET }}
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_EC2_METADATA_DISABLED: true


Confusing diffs in this file

Added a name

Moved env to the bottom for consistency

Moved inputs and steps to the env

.github/actions/gitConfig/action.yml

cristiand391

seems good overall, left some comments about things I'll test on QA.

env vars are correctly wrapped in double quotes.

notes for myself:
see if command substitution still works as input -> env var -> echo

cristiand391 · 2024-08-21T15:58:10Z

QA notes:

✅ getPreReleaseTag
✅ npmPublish
✅ create-github-release (tested with prereleases)
❌ NUTs fail:
https://github.com/salesforcecli/plugin-data/actions/runs/10492620689/job/29064652653

I think it's because the checkout happens after setup-node:
https://github.com/search?q=repo%3Aactions%2Fsetup-node%20%2FSupported%20file%20patterns%2F&type=code

✅ doesn't execute ls or $(ls) when printing from output via env var (also couldn't find any workflow that picks up user-content and tries to echo it (validatePr is already fixed)

printing output directly does run the command:
https://github.com/iowillhoit/gha-sandbox/actions/runs/10374680197/job/28722650085

…lows into ew/input-envs

fix: refactor ghas

68b466e

iowillhoit mentioned this pull request Aug 1, 2024

Remove deprecated inputs salesforcecli/cli#1823

Merged

iowillhoit commented Aug 1, 2024

View reviewed changes

chore: move output to env

944378e

iowillhoit commented Aug 1, 2024

View reviewed changes

iowillhoit added 2 commits August 1, 2024 16:33

chore: remove comment

cc12cff

chore: remove token

47e4d70

cristiand391 reviewed Aug 13, 2024

View reviewed changes

.github/actions/gitConfig/action.yml Outdated Show resolved Hide resolved

cristiand391 reviewed Aug 13, 2024

View reviewed changes

Update .github/actions/gitConfig/action.yml

da868e6

iowillhoit added 3 commits August 26, 2024 11:53

fix: checkout order

cf69de6

Merge branch 'ew/input-envs' of github.com:salesforcecli/github-workf…

1d10865

…lows into ew/input-envs

chore: optionally set env

01209f2

cristiand391 approved these changes Aug 26, 2024

View reviewed changes

iowillhoit merged commit b171803 into main Aug 27, 2024
1 check passed

iowillhoit deleted the ew/input-envs branch August 27, 2024 14:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor GHAs #124

Refactor GHAs #124

iowillhoit commented Aug 1, 2024 •

edited

Loading

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

cristiand391 Aug 13, 2024

iowillhoit Aug 1, 2024

cristiand391 Aug 13, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

iowillhoit Aug 1, 2024

cristiand391 left a comment

cristiand391 commented Aug 21, 2024 •

edited

Loading

		@@ -1,22 +0,0 @@
		# This action is only needed as long as the developer site and other docs are linking to the old sfdx.pkg file and the mac signing job is only signing the old file as well.
		# It can be deleted once those are updated to use the new name, sfdx-x64.pkg.

		- uses: salesforcecli/github-workflows/.github/actions/yarnInstallWithRetries@main
		# todo: download the macos tarball and pass that to the oclif pack:macos command

Refactor GHAs #124

Refactor GHAs #124

Conversation

iowillhoit commented Aug 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TODO: Do we need this? Isn't that why we use npx?

TODO: What does this mean? Would yarnInstallWithRetries work here?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cristiand391 left a comment

Choose a reason for hiding this comment

cristiand391 commented Aug 21, 2024 • edited Loading

iowillhoit commented Aug 1, 2024 •

edited

Loading

cristiand391 commented Aug 21, 2024 •

edited

Loading